Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOC: WIP: Convert markdown to rehype's hast at build time. #1495

Draft
wants to merge 2 commits into
base: dev
Choose a base branch
from

Conversation

tomprince
Copy link
Contributor

Originally posted by @d0sboots in #1489 (comment)

Related to that was my thought that we should probably render the MD pages to html as part of building. There's a noticable pause in loading some of those docs because of the client-side rendering it's doing. That would eliminate the need for MathJax in the docs.

I was curious to see how hard it would be to do this. At first I attempted to convert directly to html at webpack time. However, that ended up with essentially unstyled output.

This is my second attempt, which converts markdown to hast (which is the HTML ast used by react-markdown) at webpack-time. Then, at runtime, it loads the JSON hast, and tricks react-markdown into using that directly, rather than parsing the markdown at runtime.

I'll likely leave this to somebody else to turn into something that can be merged.

@tomprince tomprince force-pushed the hast-at-webpack-time branch 2 times, most recently from 577edcf to ef9471d Compare July 17, 2024 06:29
This is a demo of parsing markdown all the way to hast JSON,
and then tricking react-markdown into using that pre-parsed
hast directt, rather than parsing markdown at runtime.
@d0sboots
Copy link
Collaborator

One thing I'll note is that the speed issue largely depends on full preprocessing, since looking at the profiles showed that at least half the time was spent dealing with MathJax. (The slow pages are the ones with lots of formulas.)

@tomprince
Copy link
Contributor Author

I added a second commit which uses hast-util-to-jsx-runtime directly, rather than tricking react-markdown into using the pregenerated hast. However, that library doesn't pass all the props that react-markdown@8 does. in particular, li, ul and code blocks don't have props that we use, so those have broken styling with that commit.

Note that the prop changes are also in react-markdown@9. See https://github.com/remarkjs/react-markdown/blob/main/changelog.md#remove-extra-props-passed-to-certain-components

@tomprince
Copy link
Contributor Author

tomprince commented Jul 17, 2024

One thing I'll note is that the speed issue largely depends on full preprocessing, since looking at the profiles showed that at least half the time was spent dealing with MathJax. (The slow pages are the ones with lots of formulas.)

I've not done any profiling or benchmarking, but the mathjax processing is happening at webpack-time. The result hast-json of the SVG elements generated by mathjax.

@d0sboots
Copy link
Collaborator

One thing I'll note is that the speed issue largely depends on full preprocessing, since looking at the profiles showed that at least half the time was spent dealing with MathJax. (The slow pages are the ones with lots of formulas.)

I've not done any profiling or benchmarking, but the mathjax processing is happening at webpack-time. The result hast-json of the SVG elements generated by mathjax.

Oh, nice!

I'm still a bit confused about the fundamental issue, though: Why can't you go all the way down to HTML? I was envisioning we just embed them in an iframe or something.

@tomprince
Copy link
Contributor Author

I'm still a bit confused about the fundamental issue, though: Why can't you go all the way down to HTML? I was envisioning we just embed them in an iframe or something.

There is nothing fundamental preventing us from compiling down to html. As I mentioned, I had a version that did that (and then displayed it inline using dangerouslySetInnerHTML). It would take some work to get styling correct, and also to handle links and the corrupted text hiding advanced concepts.

If somebody is looking to do that, you can add rehype-stringify (and maybe, if using an iframe, rehype-document) as additional plugins to remark-loader. There are also a bunch of other plugins that might be useful in adding the appropriate styles.

Mostly I went with the approach I did to get something working with the correct styling with minimal effort.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants